Skip to content

Fix dde_tests: import OptimizationOptimJL for BFGS, use AutoForwardDiff#310

Merged
ChrisRackauckas merged 5 commits into
SciML:masterfrom
ChrisRackauckas-Claude:fix-dde-test-bfgs-import
Jun 20, 2026
Merged

Fix dde_tests: import OptimizationOptimJL for BFGS, use AutoForwardDiff#310
ChrisRackauckas merged 5 commits into
SciML:masterfrom
ChrisRackauckas-Claude:fix-dde-test-bfgs-import

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Problem

Master CI is red on all three tests / Core variants (Julia lts/1/pre) and the Downgrade job. All fail identically:

Core/dde_tests.jl: Error During Test
  Got exception outside of a @test
  LoadError: UndefVarError: `BFGS` not defined
  @ test/dde_tests.jl:42

Root cause

test/dde_tests.jl used BFGS and Optimization without importing the packages that provide them. It only did using DiffEqParamEstim, OptimizationNLopt.

Before the SciMLTesting v1.2 migration (#308), the old runtests.jl included every test file into the same Main module. optim_test.jl's using Optim/OptimizationOptimJL therefore leaked BFGS into shared scope, so the missing import in dde_tests.jl went unnoticed. After #308, each file runs in its own isolated @safetestset module, so the latent missing import surfaces as UndefVarError.

Fix

  1. using DiffEqParamEstim, Optimization, OptimizationOptimJL (both already test deps via [extras]/[targets]) so BFGS/Optimization resolve.
  2. Drop the dead, unused opt = Opt(:GN_ESCH, 1) line (it referenced Opt from the now-removed OptimizationNLopt import and was never used — the solve uses BFGS()).
  3. Switch the loss AD from AutoZygote() to AutoForwardDiff(). Fixing the import revealed a second latent failure: AutoZygote() gradient preparation for this DDE loss errors with MethodError: no method matching _prepare_pullback_aux in DifferentiationInterface 0.7.18. The sibling nlopt_test.jl already documents this exact issue (#zygote behaves weirdly here) and uses AutoForwardDiff(). With ForwardDiff the test actually runs and asserts the real result.

No test was skipped, broken, or loosened — the @test res.u[1] ≈ 0.5 atol = 5.0e-3 assertion is unchanged and genuinely passes.

Local verification

Ran test/dde_tests.jl inside an isolated @safetestset on Julia 1.10 with the exact CI-pinned stack (Optimization 4.8.0, OptimizationBase 2.14.0, OptimizationOptimJL 0.4.5, DifferentiationInterface 0.7.18, Zygote 0.7.10):

Test Summary:      | Pass  Total   Time
DDE Tests isolated |    1      1  20.2s

runic --check test/dde_tests.jl passes (exit 0, no diff).

(The separate Documentation job failure is unrelated — external linkcheck 403/timeout on docs.sciml.ai/Zygote/stable/ and sensitivity.sciml.ai/dev/ — and is not addressed here.)


Please ignore until reviewed by @ChrisRackauckas.


Update (additional fixes)

After fixing dde_tests.jl, the Core/Downgrade jobs surfaced the same latent missing-import bug in two more files, now also fixed in this PR:

  • test/likelihood.jl and test/test_on_monte.jl both call the qualified Optimization.OptimizationProblem / Optimization.AutoForwardDiff but only imported a re-exporting backend (OptimizationBBO / OptimizationOptimJL). using OptimizationOptimJL re-exports Optimization's exported symbols but does not bind the Optimization module name into scope, so under v1.2 isolation these errored with UndefVarError: Optimization not defined (likelihood.jl:26 was the first file to abort the Core run; test_on_monte.jl sorts after it and would have failed next). Added Optimization (already a test dep via [extras]/[targets]) to the using line of both.

Local verification (Julia 1.10, the compat floor / CI lts), each file in an isolated @safetestset:

Core/likelihood.jl isolated      | 5 Pass, 0 errors  (was: UndefVarError: Optimization not defined)
Core/test_on_monte.jl isolated   | 1 Pass, 0 errors

Both files are Runic-format clean. No assertions were skipped, broken, or loosened.

Update: the Documentation job is now also fixed here

The Documentation / Build and Deploy red was a pre-existing master failure from the OrdinaryDiffEq v7 / DifferentialEquations v8 ecosystem bump. It is now addressed in this PR (commit docs: migrate examples to current solver/EnsembleSolution API). There were three independent docs-source breakages, all fixed against the current API — no example was skipped, warnonly-ed, or loosened:

  1. verbose = falseverbose = None(). OrdinaryDiffEq v7 rejects a Bool for verbose (ArgumentError: Passing a Bool for verbose is no longer supported ... Use DEVerbosity() or a preset like Standard(), None(), etc. from SciMLLogging). None() is the no-output SciMLLogging.AbstractVerbosityPreset (the closest match for the old false). Added using SciMLLogging: None. Affects getting_started.md, generalized_likelihood.md, stochastic_evaluations.md.
  2. SDE solvers no longer re-exported. DifferentialEquations v8 was slimmed to {OrdinaryDiffEq, Reexport, SciMLBase} and no longer re-exports SRIW1/SOSRI, so stochastic_evaluations.md hit UndefVarError. Added using StochasticDiffEq: SRIW1, SOSRI (same way the package's own tests import them).
  3. EnsembleSolution indexing changed. sim[i] now returns a scalar (flat element index), not the i-th trajectory, so ensemble.md's losses[i](sim[i]) errored with MethodError: no method matching (::L2Loss)(::Float64). Changed to sim.u[i] (the i-th ODESolution).

Added SciMLLogging and StochasticDiffEq to docs/Project.toml [deps]/[compat] (both already in the resolved Manifest as transitive deps; resolution unchanged, DifferentialEquations stays at v8.0.0).

Local verification (Julia 1.12, the failing CI 1 channel): reproduced the original verbose=false ArgumentError, then ran every affected @example block end-to-end after the fix — getting_started recovers a≈1.5 (1-param) and [1.5,1.0,3.0,1.0] (multi-param); ensemble loss is exactly 0.0 on the generating sim and both Fminbox(BFGS())/BFGS() recover [1.5,1.0]; the SDE page's SRIW1/SOSRI solves and None() resolve.

ChrisRackauckas and others added 5 commits June 15, 2026 10:28
…dDiff

The DDE test referenced `BFGS` and `Optimization` without importing the
packages that provide them. Under the old `include`-everything `runtests.jl`,
`optim_test.jl`'s `using Optim`/`OptimizationOptimJL` leaked `BFGS` into the
shared `Main` scope, so the missing import went unnoticed. With the
SciMLTesting v1.2 migration each test file now runs in its own isolated
module, exposing `UndefVarError: BFGS not defined`.

Add `using Optimization, OptimizationOptimJL` (both already test deps) and
drop the dead, unused `Opt(:GN_ESCH, 1)` line.

Fixing the import revealed a second, latent failure: `AutoZygote()` gradient
preparation for the DDE loss errors with `MethodError: no method matching
_prepare_pullback_aux` in DifferentiationInterface. The sibling `nlopt_test.jl`
already documents this ("#zygote behaves weirdly here") and uses
`AutoForwardDiff()`; switching the DDE test to `AutoForwardDiff()` makes it run
and assert the real result (res.u[1] ≈ 0.5).

Verified locally on Julia 1.10 with the CI-pinned stack (Optimization 4.8.0,
OptimizationBase 2.14.0, OptimizationOptimJL 0.4.5, DifferentiationInterface
0.7.18) running the file in an isolated @safetestset: 1 Pass, 0 errors.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After the SciMLTesting v1.2 migration (SciML#308) each test file runs in its own
isolated @safetestset module, so latent missing imports that were previously
masked by the old include-everything runtests.jl now surface as UndefVarError.

`test/likelihood.jl` and `test/test_on_monte.jl` both use the qualified
`Optimization.OptimizationProblem`/`Optimization.AutoForwardDiff` but only
imported a re-exporting backend (OptimizationBBO / OptimizationOptimJL).
`using OptimizationOptimJL` re-exports Optimization's *exported symbols* but
does not bind the `Optimization` *module* name into scope, so the qualified
references errored with `UndefVarError: Optimization not defined`. This is the
same class of failure SciML#310 already fixed in dde_tests.jl.

Add `Optimization` (already a test dep via [extras]/[targets]) to the `using`
line of both files. No assertions changed.

Verified locally on Julia 1.10 (the compat floor / CI lts) by running each
file in an isolated @safetestset against the develop'd package:
  Core/likelihood.jl isolated      | 5 Pass, 0 errors
  Core/test_on_monte.jl isolated   | 1 Pass, 0 errors
Both Runic-format clean (format_string is a no-op).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Documentation job was red on master after the OrdinaryDiffEq v7 /
DifferentialEquations v8 ecosystem bump. Three independent docs-source
breakages, all fixed against the current API:

- verbose: OrdinaryDiffEq v7 rejects a `Bool` for `verbose`
  (`ArgumentError: Passing a Bool for verbose is no longer supported ...
  Use DEVerbosity() or a preset like Standard(), None(), etc. from
  SciMLLogging`). Replace `verbose = false` with `verbose = None()`
  (a SciMLLogging AbstractVerbosityPreset, the no-output preset) and add
  `using SciMLLogging: None`. Affects getting_started.md,
  generalized_likelihood.md, stochastic_evaluations.md.

- StochasticDiffEq solvers: DifferentialEquations v8 was slimmed to
  {OrdinaryDiffEq, Reexport, SciMLBase} and no longer re-exports the SDE
  solvers, so `SRIW1`/`SOSRI` in stochastic_evaluations.md errored with
  UndefVarError. Add an explicit `using StochasticDiffEq: SRIW1, SOSRI`
  (matching how the package's own tests import them).

- EnsembleSolution indexing: `sim[i]` now returns a scalar (flat
  element indexing), not the i-th trajectory, so the ensemble.md loss
  `losses[i](sim[i])` hit `MethodError: no method matching
  (::L2Loss)(::Float64)`. Use `sim.u[i]` to get the i-th ODESolution.

Add SciMLLogging and StochasticDiffEq to docs/Project.toml [deps]+[compat].
No test/example was skipped, broken, warnonly-ed, or loosened.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After the SciMLTesting v1.2 migration (SciML#308) each test file runs in its own
isolated @safetestset module, so latent missing imports that were previously
masked by the old include-everything runtests.jl now surface as UndefVarError.

`test/out_of_place_odes.jl` used `L2Loss`, `build_loss_objective`, and
`two_stage_objective` (all exported by DiffEqParamEstim) but only imported
`OrdinaryDiffEq, Test, SciMLSensitivity, Optimization, OptimizationOptimJL`.
Under the old shared-`Main` runtests, a sibling file's `using DiffEqParamEstim`
leaked those names into scope; under v1.2 isolation the file aborts the Core
run with `UndefVarError: L2Loss not defined`. This was the last file failing
all three Core variants (lts/1/pre) and the Downgrade job. Same class of
failure already fixed in this PR for dde_tests.jl, likelihood.jl, and
test_on_monte.jl.

Add `DiffEqParamEstim` to the `using` line. No assertions changed; the
`Optimization.AutoZygote()` loss path works here unchanged.

Verified locally on Julia 1.10 (CI lts / compat floor) by running the file in
an isolated @safetestset against the develop'd package:
  OOP isolated | 2 Pass, 0 errors  (was: UndefVarError: L2Loss not defined)
Runic-format clean (format_string is a no-op).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Update: fix out_of_place_odes.jl (last Core/Downgrade red)

After the previous import fixes (dde_tests / likelihood / test_on_monte), the Core (lts/1/pre) and Downgrade jobs surfaced the same latent missing-import class of failure in one more file:

Core/out_of_place_odes.jl: Error During Test
  Got exception outside of a @test
  LoadError: UndefVarError: `L2Loss` not defined in `Main.var"##Core/out_of_place_odes.jl#..."`
  @ test/out_of_place_odes.jl:26

test/out_of_place_odes.jl uses L2Loss, build_loss_objective, and two_stage_objective (all exported by DiffEqParamEstim) but only imported OrdinaryDiffEq, Test, SciMLSensitivity, Optimization, OptimizationOptimJL. Under the old include-everything runtests.jl a sibling file's using DiffEqParamEstim leaked those names into shared Main; under v1.2 @safetestset isolation the file aborts. This was the last file failing all three Core variants and Downgrade.

Fix: add DiffEqParamEstim to the using line. The Optimization.AutoZygote() loss path here works unchanged (no AD swap needed). No assertions changed/skipped/loosened.

Local verification (Julia 1.10, CI lts / compat floor), file in an isolated @safetestset against the develop'd package:

OOP isolated | 2 Pass, 0 errors   (was: UndefVarError: L2Loss not defined)

Runic-format clean (format_string is a no-op).

A scan of all test/*.jl now confirms every test file except runtests.jl (which only uses SciMLTesting) imports DiffEqParamEstim, so this was the final latent missing-import.

Please ignore until reviewed by @ChrisRackauckas.

@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 20, 2026 11:02
@ChrisRackauckas ChrisRackauckas merged commit e95f58f into SciML:master Jun 20, 2026
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants